The background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
As countries around the world transition from industrial-based economies to knowledge-based economies, it is increasingly important that we develop systems and methods for assessing the quality of knowledge-based processes, workers and their output.
In executing knowledge-based processes, knowledge workers (e.g., employees of an organization, students, teachers, professors, researchers, etc.) frequently read articles of content and make notes, conclusions, impressions, and/or perceptions (collectively as “assertions”) about what they have learned out of what they have read. Examples of assertions include capturing facts or findings, voicing opinions, asking questions and drawing conclusions. In some cases, knowledge workers will rely on these assertions for making decisions, making recommendations to decision-makers or integrating their assertions into new content they create and some cases publish/distribute.
Historically, we have relied on citation-based reference systems to assess the quality of assertions found in written content that are derived-from, or otherwise supported-by, other articles of content. Standards have been developed in various professional domains which promote consistency of creating footnotes, endnotes and bibliographies, facilitating the ability of the reader to verify assertions found in written content.
Citation-based reference systems present several limitations. They require significant time and effort by knowledge workers who create and rely on them, while introducing numerous opportunities for errors and misuses. Examples of processes citation authors have to execute include source identification, citation capture, page referencing, and formatting. Errors in any of these processes can result in the failure of the quality control system, and numerous opportunities for errors are present. These processes usually take place across multiple systems, requiring the transposing of information from one system to another system. In addition, they frequently involve the copying/pasting of information, and may involve paper-based processes which occur offline and are not electronically auditable. The citation reader has to execute several processes as well. They have to read the footnotes or endnotes, use that information to search for the source documents, acquire the source documents, navigate to the appropriate location within the source documents and make an assessment about the veracity of the source document and its author as well as the merits of the fit between the specific section in the source work and the proposition in the authored work.
Citation systems do not provide means for even capturing assessments of assertion quality such that they could be used in a system for assessing the quality and performance of knowledge-based processes, workers and their output. No means to capture assessments, manage them or use them to advance or improve knowledge work.
Efforts have been made to improve on the citation process with the help of computer technologies and the Internet. For example, hyperlinking keywords or key phrases in online articles within Wikipedia® can alleviate some of the manual work associated with citation-based references.
In another example, the “Explore Sources” feature created by DocumentCloud, an open source project sponsored by the Knight Rider foundation, provides readers of a written work direct access to the source documents and excerpts within those source documents, as disclosed in the article entitled “Explore Sources: A New Feature to ‘Show Our Work’” by Al Shaw, published on Dec. 15, 2011. While DocumentCloud reduces some of the manual work associated with creating and linking to references, it does not provide an assertion-quality assessment system—or even a means to capture an assessment.
With respect to rating, U.S. patent publication 2012/0297182 to Hawkins titled “Cipher and Annotation Technologies for Digital Content Devices”, filed Mar. 17, 2012, discloses a mechanism that allows readers to rate annotations (i.e., comments) of a document. However, the annotations described in Hawkins refer to comments made on a document, rather than independent assertion objects that are supported by or referenced to one or more documents. In addition, the rating system disclosed in Hawkins merely allows readers to indicate how much they liked the comments on the annotations and do not provide any meaningful indication of the quality of assertions made by the annotations in relation to its associated literals.
As such, knowledge enterprises and their workers are in need of a better system for assessing the quality of written works.
All publications herein are incorporated by reference to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies and the definition of that term in the reference does not apply.
The following description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
In some embodiments, the numbers expressing quantities of ingredients, properties such as concentration, reaction conditions, and so forth, used to describe and claim certain embodiments of the invention are to be understood as being modified in some instances by the term “about.” Accordingly, in some embodiments, the numerical parameters set forth in the written description and attached claims are approximations that can vary depending upon the desired properties sought to be obtained by a particular embodiment. In some embodiments, the numerical parameters should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of some embodiments of the invention are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable. The numerical values presented in some embodiments of the invention may contain certain errors necessarily resulting from the standard deviation found in their respective testing measurements.
As used in the description herein and throughout the claims that follow, the meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise.
The recitation of ranges of values herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g. “such as”) provided with respect to certain embodiments herein is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the invention.
Groupings of alternative elements or embodiments of the invention disclosed herein are not to be construed as limitations. Each group member can be referred to and claimed individually or in any combination with other members of the group or other elements found herein. One or more members of a group can be included in, or deleted from, a group for reasons of convenience and/or patentability. When any such inclusion or deletion occurs, the specification is herein deemed to contain the group as modified thus fulfilling the written description of all Markush groups used in the appended claims.